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AVERAGE CASE COMPLEXITY 
OF LINEAR MULTIVARIATE PROBLEMS 



H. WOZNIAKOWSKI 

Abstract. We study the average case complexity of a linear multivariate problem 
(LMP) defined on functions of d variables. We consider two classes of information. 
The first A"'"* consists of function values and the second A''" of all continuous linear 
functionals. Tractability of LMP means that the average case complexity is 0((l/e)'') 
with p independent of d. We prove that tractability of an LMP in A*^*"^ is equivalent 
to tractability in A^", although the proof is not constructive. We provide a simple 
condition to check tractability in A''". 

We also address the optimal design problem for an LMP by using a relation to 
the worst case setting. We find the order of the average case complexity and optimal 
sample points for multivariate function approximation. The theoretical results are 
illustrated for the folded Wiener sheet measure. 



1. Introduction 

A linear multivariate problem (LMP) is defined as the approximation of a con- 
tinuous linear operator on functions of d variables. Many LMP's are intractable 
in the worst case setting. That is, the worst case complexity of computing an e- 
approximation is infinite or grows exponentially with d (see, e.g., [9]). For example, 
consider multivariate integration and function approximation of r times continu- 
ously differentiablc functions of d variables. Then the worst case complexity is of 
order (1/e)'^/'' assuming that an e-approximation is computed using function val- 
ues. Thus, if only continuity of the functions is assumed, r = 0, then the worst 
case complexity is infinite. For positive r, if d is large relative to r, then the worst 
case complexity is huge even for modest e. In either case, the problem cannot be 
solved. 

In this paper we study if tractability can be broken by replacing the worst case 
setting by an average case setting with a Gaussian measure on the space of functions. 
The average case complexity is defined as the minimal average cost of computing an 
approximation with average error at most e. We consider two classes of information. 
The first class A''*'* consists of function values, and the second class A^" consists of 
all continuous linear functionals. 

We say an LMP is tractable if the average case complexity is O ((1/e)^) with p 
independent of d. The smallest such p is called the exponent of the problem. Under 
mild assumptions, we prove that tractability in A'^" is equivalent to tractability in 
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A'**'^ and that the difference of the exponents is at most 2. The proof of this rcsuh 
is not constructive. We provide, however, a simple condition to check tractabihty 
in A**". 

In particular, this means that multivariate integration is tractable in A^*'* and its 
exponent is at most 2. This should be contrasted with the worst case setting where, 
even for d = 1, the worst case complexity in A^**^ can be infinite or an arbitrary 
increasing function of 1/e (see [14]). Of course, intractability of multivariate inte- 
gration in the worst case setting can also be broken by switching to the randomized 
setting and using the classical Monte Carlo algorithm. 

The optimal design problem of constructing sample points which achieve (or 
nearly achieve) the average case complexity of an LMP in A^**^ is a challenging 
problem. This problem has long been open even for multivariate integration and 
function approximation. In what follows, we will use the word "optimal" modulo a 
multiplicative constant which may depend on d but is independent of e. Recently, 
the optimal design problem has been solved for multivariate integration for specific 
Gaussian measures (see [15] for the classical Wiener sheet measure, [5] for the the 
folded Wiener sheet measure, and [13] for the isotropic Wiener measure). 

In this paper, we show under a mild assumption that tractability of function 
approximation (APP) implies tractability of other LMPs. Therefore, it is enough 
to address optimal sample points for APP. Optimal design for APP is analyzed by 
exhibiting a relation between average case and worst case errors of linear algorithms 
for APP. This relation reduces the study of the average case to the worst case for 
a different class of functions. This different class is the unit ball of a reproducing 
kernel Hilbert space whose kernel is given by the covariance kernel of the average 
case measure. Similar relations have been used in many papers for approximating 
continuous linear functional; a thorough overview may be found in [11]. 

We illustrate the theoretical results for the folded Wiener sheet measure. In this 
case, an LMP is tractable and has exponent at most 2. For APP the exponents in 
A''*'* and A*^'' are the same. The exponent in A**'' was known (see [4]), whereas the 
exponent in A^**^ was known to be at most 6 (see [3]). Tractability of APP for the 
folded Wiener sheet measure is in sharp contrast to intractability of APP for the 
isotropic Wiener measure; see [13]. 

Tractability of APP in the average case setting is significant, since it is known 
that the randomized setting does not help (sec [12]). Thus, unlike multivariate 
integration, intractability of APP in the worst case setting cannot be broken by the 
randomized setting. 

APP has been studied in A"^^' for = 1 in [2, 6]. For > 1, it was shown in 
[4] that the number of grid points needed to guarantee an average error e depends 
exponentially on d. Of course, 0{e~'^~^) sample points are enough to compute an 
e-approximation, S > 0. Hence, grid points are a poor choice of sample points. 

In [4], the average case complexity of APP in A*^" was found, and it was conjec- 
tured that the average case complexity in A^**^ is of the same order. We prove that 
this is indeed the case. 

Optimal design for APP is solved by using a relation to the worst case setting in 
the reproducing kernel Hilbert space H. For the folded Wiener sheet measure, H 
is a Sobolcv space of smooth nonperiodic functions which satisfy certain boundary 
conditions. 

APP in the worst case setting has been studied in this Sobolev space additionally 
assuming periodicity of functions in [7, 8] (see also [10] for d = 2). It was proven 
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that hyperbolic cross points are optimal sample points. Hyperbolic cross points are 
defined as a subset of grid points whose indices satisfy a "hyperbolic" inequality. 
Approximation of periodic functions by trigonometric polynomials that use Fourier 
coefficients with these hyperbolic cross indices was first studied in [1]. 

For the nonperiodic case, optimal sample points for APP in the average case 
setting are derived from hyperbolic cross points, and the average case complexity 
is given by 

comp-8(£;APP) = 9 (e-^/^'-^+i/^) ( log ^'*-'^^''"""+'^/^'^-'"+'/'^) , 

with rmin = iiiini<j<rfrj, where /('■i' - >''<j) is continuous and where k* denotes the 
number of equal to rmin. An optimal algorithm is given by a linear combination 
of function values at sample points derived from hyperbolic cross points. 
Proofs of the results reported here can be found in [16] . 

2. Linear multivariate problems 

A linear multivariate problem LMP = {LMP^} is a sequence of LMP^ = {F, /x, 
G, S, A) may depend on d. We now define them in turn. 

Let F be a separable Banach space of functions f : D ^ 'R, F C £2(0). 
Here, L* c R , and its Lebesgue volume 1{D) is in (0,+oo). We assume that all 
X(/)=/(x) are in F*. 

The space F is equipped with a zero mean Gaussian measure /i. Let i?^ be the 
covariancc kernel of ^, i.e., Rfj,{t, x) = f(t) f{x) /i (df) for t,x G D. 

Let S : F ^ G he a. continuous linear operator, where G is a separable Hilbert 
space. Then u = fj,S~^ is a zero mean Gaussian measure on the Hilbert space G. 
Its covariance operator Ci^ = C* > and has a finite trace. 

Finally, A is either A'^" = F* or A"" which consists of L{f ) = f{x),\lf G F, for 

xeD. 

Our aim is to approximate elements S{f) by U{f). The latter is defined as 
follows. Information about / is gathered by computing a number of L{f), where 
LgA, 

N{f) = [Liif),L2{f),...,LM V/eF. 

The choice of Li and n = n{f) may depend adaptively on the already computed 
information (see [9, Chapter 3]). Knowing y = N{f), we compute U{f) = 4'{y) for 
some (j) : N{F) — > G. The average error of U is defined as 

e-^(C/) = i^JJS{f)-U{f)f^^{df) 

To define the average cost of U, assume that each evaluation of L E A and 

f G F, costs c = c{d) > 0. Assume that we can perform arithmetic operations 
and comparisons on real numbers as well as addition of two elements from G and 
multiplying an element from G by a scalar; all of them with cost taken as unity. 
Usually c > 1. 

For [/(/) = (j){N{f)), let cost{N,f) denote the information cost of computing 
y = N{f). Clearly, we have cost{N,f) > cn{f). Let ni(/) denote the number 
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of operations needed to compute <j){y) given y = N{f). (It may happen that 
ni(/) = +00.) The average cost of U is then given as 

cost-^s([;) = ^(cost(Ar,/) + m{f))fx{df). 

The average case complexity of LMP^ is the minimal cost of computing e-approxi- 
mations, 

comp'*^e(£.LMPd) = m{{cose''^{U) : U such that e^^^{U) < s}. 

To stress the dependence on certain parameters in comp'^^^{s; LMP^), we will some- 
times list only those. Obviously, comp^^'^{e, d, A'^") < comp'^^s(£; A^'*^). Wc show 
that the average case complexity functions in A'''^ and A''*'^ are usually closely re- 
lated. 

3. Tractability of linear multivariate problems 
An LMP — {LMP(j} is called tractable if there exists p > such that for all d 

(3.1) comp^^s(£.LMPd) = 0{ce'P). 

The constant in the big O notation may depend on d. The infimum of the numbers 
p satisfying (3.1) is called the exponent p* = p*(LMP). To stress the role of the 
class A, we say that an LMP is tractable in A iff (3.1) holds for A. 

In what follows, by multivariate function approximation we mean APP = LMP 
with the embedding S{f) = Id{f) = f & G = L^iD), where the norm in 1^2 (-D) is 
denoted by || • 

We assume that for all d there exist Ki = Ki(d), i = 1,2, such that 

(A.l) ||^(,/)|| < i^ill/lld, V/eF, 

(A.2) \\Rf.{;-)\\L^iD) < K2. 

Theorem 3.1. Suppose (A.l) and (A.2) hold. 

(i) Tractability of LMP in A^**^ is equivalent to tractability of LMP in A*'' since 

comp^^8;(£.^^A^") = 0{cs-p^''^) implies comp^^s(£. a^'^) = 0{ce-P^'^'>-^). 

(ii) Let Xi{d) be the ordered eigenvalues of the covariance operator of iiS~^. LMP 
is tractable in A*'' iff there exists a positive number a such that for all d, 

(3.2) Ai((i) = 0(n-2"), as n^+oo. 

i=n+l 

The exponent of LMP is p* = 1/ sup{a : a of (3.2)}, and p* = +oo if there is no 
such a. 

(iii) Tractability of APP in A with exponent p* implies tractability of an LMP 
in A with exponent at most p* provided LMP differs from APP only by the choice 
ofS. 

We stress that the proof of Theorem 3.1 is not constructive. The exponents in 
A*^'' and A^**^ may differ by at most 2. The constant 2 is sharp. Indeed, for the 
integration problem with the isotropic Wiener measure, the exponent in A^**^ is 2 
(see [13]), and, obviously, the exponent in A"*'' is zero. 
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4. Relation to worst case 

Due to (iii) of Theorem 3.1, it is enough to analyze multivariate function ap- 
proximation APP = {APPd} with APPd = {F,/x,X2(I>),/d,A'^"}. The aver- 
age case errors of APP are related to worst case errors of the same Id restricted 
to a specific subset of F. This specific subset of F is the unit ball BH^j^ of a 
reproducing kernel Hilbert space H^. The space is the completion of finite- 
dimensional spaces of the form 

span {Rfii-, xi), R^{-, X2), Xfc)) . 

1/2 

The completion is with respect to || • ||^ = (•, •)^' , where x), R{-, t)) ^ = R{x, t). 
Consider a linear U which uses sample points Xj. That is, we have U{f) = 
f{^j)9j^ where gj € Loo{D). It is easy to show that 

e-s([7) = e-s([/;APPrf) = (^Jjh*{;x)\\ldx 

where h*{-,x) = R^i{-,x) - Y.''j^i9j{x) R^{-,x) e H^,. 

Consider now the same U for multivariate function approximation in the I/qo {D) 
norm 

Appwor ^ {BH^,LUD),Id,A'"'} 

in the worst case setting. We now assume that i?^ is a subset of (D) and that 
the embedding Id maps iJ^ into Loo{D). The worst error of U is equal to 

e-(C/;APPr) =sup{||/-C/(/)||^^(^^: < l}. 

It is easy to show that e^°''{U;APP^°'') = esssup^-gj^ ||/i*(-,a;)||^, which yields 

(4.1) e^^s([/.APPrf) < vTCD)e™'(f/;APP7^), 

where 1{D) is the Lebesgue volume of D. 

5. Application for folded Wiener sheet measures 

We assume that fj, is the folded Wiener sheet measure (see [4]). That is, D = 
[0, l]'^ and F is the space of times continuously differentiable functions with 
respect to Xi which vanish with their derivatives at points with at least one com- 
ponent equal to zero. The norm of F is the sup norm on (n, . . . , r^) derivatives. 
The covariance kernel i?^ of ^ is 

Observe that R^{t,t) < 1 and (A. 2) holds with K2 < 1. 

The space if^ consists now of functions / of the form (see [5]) 

f (x ■ — t 

f(^)= II— — r^Hti,t2,...,td)dtidt2---dtd, yxGD, (l)GL2iD). 
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The inner product of i?^ is = /(n,...,ra) ^(ri,...,r^)(^) 

Average case errors for APP^ can be bounded (see (4.1)) by analyzing the worst 
case of 

Appwor ^ {BH^,L^iD),Id,A'"'}. 

Let Wo be a subspace of of periodic functions for which >■■■'*<!) (i) = for 
all ij < rj and all t from the boundary of D. Multivariate function approximation 
for the unit ball of Wq in the worst case setting has been analyzed by Temlyakov in 
[7, 8]. He constructed sample points xj and functions aj such that for Tn{f,x) = 
Ej=i f{xj) O'iix) we have 

(5.1) 11/ - T„(/, .)|k»(i,) = 0(n-('--+V2) (logn)('=*-i)('--+i)), 

where Tmin = minjrj : 1 < i < d} and k* = card({7 : rj — rmin})- 

The sample points Xj are called hyperbolic cross points and the functions Uj are 

obtained by linear combinations of the de la Vallee-Poussin kernel. 

To extend Tcnilyakov's result to nonpcriodic functions, define for / from BH^ 

g{x) = f{h{x)), VxeA 

where h{x) = (^h{x-i),h{x2), ■ ■ ■ ,h{xd)) and h{u) = Au{l — u), Vue[0, 1]. 

Observe that g is periodic and enjoys the same smoothness as /; that is, g € Wq. 
There exists a constant K = K{d, f) such that < K. Define 

U:{f,t) = Tn{g,h-\t)), 

where h-'^{t) = - s/T^), . . . , |(1 - s/T^)), t G D. We have 

n n 

(5.2) U:{f,t) = Y.f{h{xj)) aj{h-\t)) = J2f{x*)h*{t), 

where x* = h{xj), with a hyperbolic cross point xj, and h*{t) = aj(h~^{t)). 
It is possible to check that for all / from BHf^ we have 

(5.3) 11/ - U:{f,-)\\L^^n) = 0(n-('--"+V2) (log „)('=* "DC'-^+D). 
Prom (5.3) and (4.1) we conclude that 

(5.4) C0mp^^S(£.APPrf) = C»(ce-l/(r-mi„+l/2)(iyg i/£)(fe*-l)(r„.„+l)/(r„.„+l/2))^ 

Clearly, comp'^^s^gj APP^) is bounded from below by the corresponding average 
case complexity in the class A'''^ The latter was determined in [4]. These two 
average case complexity functions differ by at most a constant. Thus, the O in 
(5.4) can be replaced by Q. Furthermore, the linear approximation U* given by 
(5.2) is optimal, i.e., U* computes an £-approximation with the average cost (c+2)n 
which is minimal, modulo a constant, if 



(5.5) n = 0(e-V(rn>i„ + l/2) (logi/£)(fc*-l)(r„,„ + l)/(r„.„+l/2)-)_ 
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Theorem 5.1. For APRhe average case complexity functions comp'^^^{s,d,A^^'^) 
and comp^'^^{e;d,A'^^^) differ at most by a constant and 

comp*^s(£;rf,A^*<i) = e(c£-i/(''-"'+i/2) (log l/e)('=*-i)('-n>in+i)/(r„i„+i/2))_ 

The linear U* given by (5.2) which uses n sample points derived from the hyperbolic 
cross points with n given by (5.5) is optimal in the classes A^**^ and A*". 

Prom Theorem 5.1 we have that APP is tractable in A^**^ since l/(rmin+l/2) < 2. 
The exponent of APP is the same in A^'' and A^*''. Since rj may depend on d, we 

have 

p*(A"*<i) = (1/2 + min{rj(rf) : j = 1, 2, . . . , d and d = 1, 2, . . . })"^ < 2. 

Obviously, any LMP which satisfies (A.l) and which is equipped with the folded 
Wiener sheet measure is tractable and has exponent at most p*(A^*'^) < 2. 
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